Adaptive Gaussian Kernel SVMs

We consider binary classification using Support Vector Machines with Gaussian kernels: KΣ(xi, xj) = e −(xi−xj)′Σ−1(xi−xj) and address the problem of selecting a covariance matrix Σ which gives good classification performance. As with other nonparametric classification methods based on distances between data points (such as nearest neighbor and Parzen methods), the choice of distance function (Σ) can be crucial to the success of learning. A good choice of Σ accounts for differences in the scales of different features (coordinates of the input vectors x), can remove global correlations between features and can adapt to the fact that some features may be much more informative about the class labels than others. We suggest an approach which ties training the SVM and choosing the distance function, as opposed to the more standard practice of feature normalization as a separate pre-processing step.